This invention relates to programmable multispectral filters that incorporate time delay and integrate electronics. The invention further relates to techniques for manipulating the data collected therefrom to identify scanned objects and to the specifics of the improved time delay and integrate (TDI) electronics themselves.
Spectroscopy is the discipline that analyzes the various spectral components of light emanating from a scene to determine what is in the scene or how it is acting in its environment. The light coming from the scene can be created in many different ways, but the immediate applications of the present invention will be concerned mostly with light from the sun or other light source that reflects off the materials in the scene that is then collected and processed by the sensor of this invention, although thermal imaging of infrared energy emanating from the scene is also of interest. By emphasizing the data containing the spectral content unique to a particular target or aspect of the scene, one can highlight that target or aspect and remove much of the clutter arising from the background environment. Much of the work in multispectral imaging has been done in the context of remote sensing by satellite-born sensors, although use of these processes is not limited thereto. The analyzed content of the remotely sensed images is useful in areas including agriculture, meteorology, oceanography, geological exploration, and various national security missions. Spectral imaging sensors have been shown to provide information far superior to that of conventional panchromatic images in many of these applications. These imagers are not limited to satellite applications and, as such, have terrestrial uses in the medical and manufacturing fields as well.
As the technology to build the photodetector arrays that measure the strength of the light in a particular spectral bin has improved, the number of channels (spectral bins) that can be sensed for a particular sample point (or pixel) has increased dramatically over the last few years. However, the light coming from the target/background is fixed, and as one increases the number of spectral channels per pixel, the signal to noise ratio in any one channel will decrease. Also, the data rates of spectral imaging sensors (10 Mbytes/sec, or more) stress the limits of the electronics systems, including the onboard data storage, the downlink bandwidth, and the earthbound image analysis system. The newest conventional multispectral sensors are called hyperspectral imagers (HSI). These sensors can record hundreds of spectral channels for each of the pixels in its array, with a typical array containing hundreds or thousands of pixels. A pixel herein is typically the patch on the ground that defines the minimum resolution of the system in which to look for a target. An HSI system offers the maximum of flexibility for post-collection analysis of the multispectral data but at an immense price in terms of data that needs to be transmitted, stored and processed.
The following references teach various approaches for collecting and processing multispectral data. U.S. Pat. No. 4,790,654 to Clarke discloses a programmable multispectral filter having means to receive a multispectral image, means to disperse the image into multiple spectral components, means to modulate the strength of the light in each spectral component, and means to reflect the modulated component back to the dispersing element for recombination of the multiple spectral components into a filtered whole image. The system can split the dispersed light into two separate channels by polarization for separate modulation in each channel. However, its optics are quite primitive. The spectral modulation is done at the pupil plane, which restricts its use to very small images with very few pixels. Although two channels can be processed at once, there is no mention of using spectral basis vectors that are developed by differencing two orthogonal channels as the means for modulating the light in the spectral bands in each channel. No reason is given for having a two channel capability, presumably one uses one channel to look for one thing and the other channel to look for another thing.
U.S. Pat. No. 5,379,065 to Cutts discloses selecting wavelengths of light that are transmitted using a spectrally agile filter (SAF). A specific embodiment of an SAF is an acousto-optic (AO) cell, where the dynamic grating in the AO cell is tuned to diffract only the wavelengths of interest. The detector is a charge coupled device (CCD) array operating in the shift-and-add mode, also known as the Time Delay and Integrate (TDI) mode. This is a two-dimensional detector that reads out only one line of pixels at a time.
U.S. Pat. No. 5,410,371 to Lambert discloses an AO tunable filter system for selecting wavelengths, one at a time. This system performs hyperspectral imaging, but not all of the wavelengths are simultaneously read; therefore, relatively longer data collection times are required than for the Cutts system.
U.S. Pat. No. 5,424,543 to Dombrowski et al discloses a method of taking hyperspectral data of a fixed scene, i.e., one for which high speed imaging is not required. A two-dimensional image is viewed serially using a series of narrow band filters such that many frames of the same image are viewed through different spectral filters.
U.S. Pat. No. 5,504,575 to Stafford discloses a spatial light modulator spectrometer. The spectrometer has collimating means, dispersing means to separate the light assigned to a particular pixel into its various spectral components, a multiplicity of spatial light modulators acting upon the dispersed light from each pixel, and recombination means to refocus the individual, now-modulated spectral components of light back into the individual pixels from whence they came. The spatial light modulators here are digital micromirrors, labeled therein as deformable mirror devices. This is a single channel spectrographic system only, not an imager.
The last references disclose two airborne systems that can collect 128-256 spectral components for each pixel scanned. These are (1) xe2x80x9cAVIRISxe2x80x9d (Airborne Visible-InfraRed Imaging Spectrometer)xe2x80x94see W. M. Porter, H. T. Enmark, xe2x80x9cA System of the Airborne Visible/ Infrared Imaging Spectrometer (AVIRIS)xe2x80x9d, SPIE, Vol. 834, Imaging Spectroscopy II, 1987 and W. M. Porter, T. G. Chrien, E. G. Hansen, Ch. M. Sature, xe2x80x9cEvolution of the Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) Flight and Ground Data Processing Systemxe2x80x9d, SPIE, Vol. 1298,1990, pp. 11-17); and (2) xe2x80x9cHYDICExe2x80x9d (Hyperspectral Data Image Collection Experiment)xe2x80x94see L. J. Rickard, R. W. Basedow, E. Zalweski, P. Silverglate, and M. Landers, xe2x80x9cHYDICE: An Airborne System for Hyperspectral Imaging,xe2x80x9d SPIE, Vol. 1937, Imaging Spectrometry of the Terrestrial Environment, 1993, p. 173 and R. W. Basedow, W. S. Aldrich, K. A. McVey, xe2x80x9cHYDICE System Performance: An Update,xe2x80x9d SPIE, Vol. 2821; Hyperspectral Remote Sensing and Applications, 1996, Paper # 2821-09. Both AVIRIS and HYDICE require significant digital post-processing of the conventional spectral data to identify the materials scanned.
Although these above references demonstrate the progress that has been made in multispectral and hyperspectral imaging, there remains a need in the art from an even more advanced and efficient means of collecting and processing multispectral data for target identification.
This invention presents a new system and method for optically processing hyperspectral data on a pixel-by-pixel basis and providing for utilization of all n spectral bins for each pixel, as necessary, to emphasize a particular aspect or aspects of a scanned scene or to provide an indication or not of whether a target is present in an imaged scene. Although the mathematics behind this technique have been known for some time and have been used for post-collection electronic processing of hyperspectral data, they are applied in a new way herein to diminish, if not eliminate, the need to collect, store, and transmit for electronic processing the entire hyperspectral data set for an image scene.
The present system relies in part upon a new type of time delay and integrate (TDI) electronics that provides for variable gain and switching off of selected channels. The variable gain TDI can be employed at the focal plane of the optical system, and by doing so can avoid the need for many of the optical components used in the sister patent application referenced above. The variable gain aspect of the TDI electronics can be rapidly reconfigured to adjust the sensor system to a different attenuation levels required by a new spectral basis vector, as discussed below. This ability to change the system response on the fly, as it were, is especially valuable in satellite-based sensor systems. This new type of TDI electronics may also find applications beyond those related to the disclosed multispectral imaging system.
The present system also relies upon a spectral basis vector to emphasize the desired signal returns and to de-emphasize the undesired signals by taking the dot product of the radiance vector, consisting the magnitudes of the radiation in all the spectral bins for a particular pixel, and the spectral basis, which acts to filter out the undesired signals. The spectral basis vector is the result of an affine transformation that is performed on a training set of data and has both positive and negative components. The positive and negative components can be utilized separately in different channels, or they can be transformed again to an all positive modified spectral basis vector that can be used in a single channel. The spectral basis vector is explained in more detail below.
This invention reduces the dimension of the spectral data to a few channels per pixel from the hundreds required in a classic hyperspectral imaging system (HIS). It also significantly improves the signal-to noise (S/N) ratio for a given computation while reducing the need for high data rate systems with their concomitant weight and complexity burdens. When used in mapping or geophysical applications, these two improvements allow the programmable hyperspectral sensor of the present invention to search huge areas rapidly, yet with high resolution. When used in other applications, such as medicine, the improvements permit near real-time identification and analysis of targets in data-rich environments. Finally, it uses the encoded spectra directly thus simplifying classification and quantification.